Implementation of Delta Compression

A Matlab simulation is carried out to verify the compression ratio analysis. Packet Xk and V are generated as two i.i.d. random sequences that follow a discrete uniform distribution between 0 and 255 with a packet length of 1,500 bytes. Packet Xk+1 is generated according to the simplified content generation model. Considering Xk and Xk+1 as two byte strings, our lossless delta compression algorithm is designed as follows. Between Xk+1 and Xk, Xk+1 is regarded as a string with many substrings copied from Xk and new bytes. Each substring is coded by its length and location in Xk. No assumptions about the format or alignment of the input data are made. The flowchart for our compression scheme is depicted in Figure 1. Essentially, this can be broken down into two steps: decomposing Xk+1 into substrings copied from Xk; coding the length and location of each substring i (leni, positioni) by a number of bits. In Step 1, obtaining the optimal partition for Xk+1 is a challenging topic. Several methods have been proposed to increase the searching speed and efficiency. For example, a hash function can be used for searching the longest matched substring. However, for short packets (say less than 1,500 bytes) or non-real time compression, hash tables are not necessary. One can simply search the current byte Xk+1(i) in Xk directly and look for the longest match. In Step 2, because leni cannot be guaranteed to be larger than 1 (otherwise a matched substring cannot be found), a modified substring coding scheme is used by adding an index bit: a single byte is coded by 9 bits (L1 = 1.125 bytes) with 1 bit for index and 8 bits for data content; a matched substring is coded by 14 bits (L2 = 1.75 bytes) with 1 index bit, 8 bits for length, and 5 bits for location. In Figure 2, the compression ratio for our lossless delta compression algorithm as well as the entropy curves are plotted for different p, which represents the degree of similarity between Xk+1 and Xk. From Figure 4, it is clear that the delta compression scheme can only be applied for larger values of p between 0.45 and 1 (i.e., 55% of the time). Intuitively, this makes sense since larger compression gains are obtained when there is large similarity between adjacent packets. Additional coding methods (e.g., Huffman coding) can be used to reduce these overheads associated with fixed length coding.